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Abstract 

Ordinary polytopes were introduced by Bisztriczky as a (nonsimpli- 
cial) generalization of cyclic polytopes. We show that the colex order 
of facets of the ordinary polytope is a shelling order. This shelling 
shares many nice properties with the shellings of simplicial polytopes. 
We also give a shallow triangulation of the ordinary polytope, and 
show how the shelling and the triangulation are used to compute the 
toric h-vectOT of the ordinary polytope. As one consequence, we get 
that the contribution from each shelling component to the /i-vector is 
nonnegative. Another consequence is a combinatorial proof that the 
entries of the /i-vector of any ordinary polytope are simple sums of 
binomial coefficients. 



1 Introduction 
1.1 Motivation 

This paper has a couple of main motivations. The first comes from the 
study of toric /i-vectors of convex polytopes. The /i-vector played a crucial 
role in the characterization of face vectors of simplicial polytopes |51 1121 113j . 
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search Center "Mathematics for Key Technologies" (FZT86) and the Research Group 
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In the simplicial case, the /i-vector is hnearly equivalent to the face vector, 
and has a combinatorial interpretation in a shelling of the polytope. The 
/i-vector of a simplicial polytope is also the sequence of Betti numbers of 
the associated toric variety. In this context it generalizes to nonsimplicial 
polytopes. However, for nonsimplicial polytopes, we do not have a good 
combinatorial understanding of the entries of the /i-vector. 

The definition of the (toric) /i-vector for general polytopes (and even 
more generally, for Eulerian posets) first appeared in jjl]. Already there 
Stanley raised the issue of computing the /i-vector from a shelling of the 
polytope. Associated with any shelling, Fi, F2, . . . , -F„, of a polytope P is 
a partition of the faces of P into the sets Gj of faces of Fj not in Ui^jFi. 
The /i- vector can be decomposed into contributions from each set Qj . When 
P is simplicial, the set Gj is a single interval [Gj,Fj] in the face lattice of 
P, and the contribution to the /i- vector is a single 1 in position \Gj\. For 
nonsimplicial polytopes, the set Qj is not so simple. It is not clear whether 
the contribution to the /i-vector from Qj must be nonnegative, and, if it 
is, whether it counts something natural. (Tom Braden |S] has announced 
a positive answer to this question, based on 0^^.) Another issue is the 
relation of the /i- vector of a polytope P to the /i- vector of a triangulation of 
P. This is addressed in 0} I15j. 

A problem in studying nonsimplicial polytopes is the difficulty of gener- 
ating examples with a broad range of combinatorial types. Bisztriczky jjj 
discovered the fascinating "ordinary" polytopes, a class of generally non- 
simplicial polytopes, which includes as its simplicial members the cyclic 
polytopes. These polytopes have been studied further in jSHHIH. In par- 
ticular, in 3 , it is shown that ordinary polytopes have surprisingly nice 
/i-vectors, namely, the /i-vector is the sum of the /i-vector of a cyclic poly- 
tope and the shifted /i-vector of a lower-dimensional cyclic polytope. These 
/i-vectors were calculated from the flag vectors, and the calculation did not 
give a combinatorial explanation for the nice form that came out. So we 
were motivated to find a combinatorial interpretation for these /i- vectors, 
most likely through shellings or triangulations of the polytopes. 

This paper is organized as follows. In the second part of this introduction 
we give the main definitions. The brief Section 2 warms up with the natural 
triangulation of the multiplex. Section 3 is devoted to showing that the 
colex order of facets is a shelling of the ordinary polytope. The proof, while 
laborious, is constructive, explicitly describing the minimal new faces of 
the polytope as each facet is shelled on. We then turn in Section 4 to h- 
vectors of multiplicial polytopes in general, and of the ordinary polytope 
in particular. Here a "fake simplicial /i-vector" arises in the shelling of the 
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ordinary polytope. In Section 5, the triangulation of the multiplex is used 
to triangulate the boundary of the ordinary polytope. This triangulation 
is shown to have a shelling compatible with the shelling of Section 3. The 
shelling and triangulation together explain combinatorially the /i-vector of 
the ordinary polytope. 

Finally, a comment about the title of this paper. Bisztriczky named 
these polytopes "ordinary polytopes" to invoke the idea of ordinary curves. 
The name is, of course, a bit misleading, as it is applied to a truly extraor- 
dinary class of polytopes. We feel that these polytopes are extraordinary 
because of their special structure, but we hope that they will also turn out 
to be extraordinary for their usefulness in understanding general convex 
polytopes. 

1.2 Definitions 

For common polytope terminology, refer to |16j . 

The toric h-vector was defined by Stanley for Eulerian posets, including 
the face lattices of convex polytopes. 

Definition 1 ((Sj) Let P be a (d — l)-dimensional polytopal sphere. The 
/i-vector and 5- vector of P are encoded as polynomials: h{P, x) = J2i=o hix'^~' 
and g{P,x) = J2i=o^ 9ix\ with the relations go = ho and gi = hi — for 
1 < i < d/2. Then the /i-polynomial and (7-polynomial are defined by the 
recursion 

1. 5(0, x) = h{ib,x) = 1, and 

2. h{P,x) = 9{G,x){x - l)'^-i-dimG_ 

G face of P 
G^P 

It is easy to see that the /i- vector depends linearly on the flag vector. 
In the case of simplicial polytopes, the formulas reduce to the well-known 
transformation between /-vector and /i- vector. 

Definition 2 ([16j) Let C be a pure d-dimensional polytopal complex. If 
d = 0, then a shelling of C is any ordering of the points of C. If d > 0, then 
a shelling of C is a linear ordering Fi, F2, ■ ■ ■ ■, Fg oi the facets of C such that 
for 2 < j < s, Fj n (Uj<jFj) is nonempty and is the union of ridges of C that 
form the initial segment of a shelling of Fj. 

Definition 3 ([2j) A triangulation A of a polytopal complex C is shallow 
if and only if every face u of A is contained in a face of C of dimension at 
most 2 dim cr. 
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Theorem 1.1 ([2]) If A is a simplicial sphere forming a shallow triangu- 
lation of the boundary of the convex d-polytope P, then h{A,x) = h{P,x). 

Note: in [2] Theorem 4 gives h{P, x) = h{A, x) for a shahow subdivision A 
of the soUd polytope P. The proof goes through for shahow subdivisions of 
the boundary, because it is based on the uniqueness of low-degree acceptable 
functions Tl", which holds for lower Eulerian posets. 

Definition 4 ([6j) A d-dimensional multiplex is a polytope with an ordered 
list of vertices, xq, xi, . . . , x„, with facets Fq, Fi, . . . , F„ given by 

Fi = conv{xi^d+i, Xi-d+2, ■ ■ ■ ,Xi-i,XiJ^i,Xi+2, ■ ■ ■,Xi+d^i}, 
with the conventions that Xi = xq if i < 0, and Xj = if i > n. 

Given an ordered set V = {xo,xi, . . . a subset Y V is called a 

Gale subset if between any two elements oiV\Y there is an even number of 
elements of Y. A polytope P with ordered vertex set y is a Gale polytope 
if the set of vertices of each facet is a Gale subset. 

Definition 5 ([7J) An ordinary polytope is a Gale polytope such that each 
facet is a multiplex with the induced order on the vertices. 

Cyclic polytopes can be characterized as the simplicial Gale polytopes. Thus 
the only simplicial ordinary polytopes are cyclics. In fact, these are the 
only ordinary polytopes in even dimensions. However, the odd-dimensional, 
nonsimplicial ordinary polytopes are quite interesting. 

We use the following notational conventions. Vertices are generally 
denoted by integers i rather than by Xj. Where it does not cause con- 
fusion, a face of a polytope or a triangulation is identified with its ver- 
tex set. Interval notation is used to denote sets of consecutive integers, 
[a,b] = {a, a + 1, . . . , 6 - 1, b}. 

2 Triangulating the multiplex 

Multiplexes have minimal triangulations that are particularly easy to de- 
scribe. 

Theorem 2.1 Let iVf^'" be a multiplex with ordered vertices 0, 1, . . . , n. 
For < i < n — d, let Ti be the convex hull of [i,i + d]. Then M'^'" has a 
shallow triangulation as the union of the n — d + 1 d-simplices Ti. 
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Proof: The proof is by induction on n. For n = d, the multiplex M ' 
is the simplex Tq itself. Assume M*^'" has triangulation into simplices Tj, 
< i < n — d. Consider the multiplex M"^'""*"^ with ordered vertices 0, 1, 
. . . , n. Then M'^'""''^ = conv(M'^'" U {n + 1}), where n + 1 is a point beyond 
facet Fn of M'^'"', beneath the facets Fi for < i < n — d+1, and in the affine 
hulls of the facets Fi for n-d + 2 < i < n-1. (See 0.) Thus, M'^'"-^^ is the 
union of M'^''" and conv(i^„ U {n + 1}) = T^+i-d, and Af^'" n r„+i_rf = F„. 
By the induction assumption, the simplices Tj, with 0<i<n + l — d, form 
a triangulation of M'^'"^-^. 

The dual graph of the triangulation is simply a path. (The dual graph 
is the graph having a vertex for each d-simplex, and an edge between two 
vertices if the corresponding d-simplices share a {d — l)-face.) The ordering 
To, Ti, T2, . . . , Tn-d is a shelling of the simplicial complex that triangulates 
M*^'". So the /i-vector of the triangulation is (1, n — d, 0, 0, . . .). This is the 
same as the (7- vector of the boundary of the multiplex, which is the /i-vector 
of the solid multiplex. So by j2j, the triangulation is shallow. □ 

Note, however, that for n > d + 2, Af^'" is not weakly neighborly (as 
observed in [^). This means that it has nonshallow triangulations. This 
is easy to see because the vertices and n are not contained in a common 
proper face of M'^'^. 

Consider the induced triangulation of the boundary of M*^'". For no- 
tational purposes we consider Tq and T„ separately. All facets of Tq ex- 
cept are boundary facets of Af^'". Write Tq\q = [0, d — 1] = Fq, and 
ro\j = [0, d\ \ {3} for 1 < j < d - 1. Write Tn-d\n = [n-d+l,n]= F„, and 
Tj^\j = \n — d,n] \ {j} for n — d+\<j<n — 1. For l<i<n — d — 1, the 
facets of Ti are Tj\j = [i,i + d]\{j}. Two of these facets (j = i and j = i + d) 
intersect the interior of Af^'". For 1 < j < n — 1, the facet Fj is triangulated 
by Ti\j for j — d+ 1 < i < j — 1 (and < i < n — d). The facet order Fq, Fi, 
. . . , Fn, is a shelling of the multiplex Af^'". The {d— l)-simplices Ti\j in the 

order Tq\q, Tq\i, Tq\2i ^1\2i • • • 1 ^n-d-l\n-2) ^n-d\n-2) ^n-d\n-li ^n-d+l\n 

(increasing order of j and, for each j, increasing order of i), form a shelling 
of the triangulated boundary of Af^'". 

3 Shelling the ordinary polytope 

Shelling is used to calculate the /i- vector, and hence the /-vector of simplicial 
complexes (in particular, the boundaries of simplicial polytopes). This is 
possible because (1) the /i-vector has a simple expression in terms of the 
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/-vector and vice versa; (2) in a shelling of a simplicial complex, among the 
faces added to the subcomplex as a new facet is shelled on, there is a unique 
minimal face; (3) the interval from this minimal new face to the facet is a 
Boolean algebra; and (4) the numbers of new faces given by (3) match the 
coefficients in the /- vector //i- vector formula. These conditions all fail for 
shellings of arbitrary polytopes. However, some hold for certain shellings of 
ordinary polytopes. 

As mentioned earlier, noncyclic ordinary polytopes exist only in odd 
dimensions. Furthermore, three-dimensional ordinary polytopes are quite 
different combinatorially from those in higher dimensions. We thus restrict 
our attention to ordinary polytopes of odd dimension at least five. It turns 
out that these are classified by the vertex figure of the first vertex. 

Theorem 3.1 (O E]) For each choice of integers n > k > d = 2m +1 > 5, 

there is a unique combinatorial type of ordinary polytope P = P'^-'^'" such 
that the dimension of P is d, P has n + 1 vertices, and the first vertex of 
P is on exactly k edges. The vertex figure of the first vertex of P"'-^'" is the 
cyclic {d — l)-polytope with k vertices. 

We use the following description of the facets of P'^^^'^ by Dinh For 
any subset X C Z, let retn(X) (the "retraction" of X) be the set obtained 
from X by replacing every negative element by and replacing every element 
greater than n by n. Let Xn be the collection of sets 

X= [i,i + 2r-l]uyu[i + A;,i + /c + 2r-l], (1) 

where i € Z, 1 < r < m, y is a paired {d — 2r — l)-element subset of 
[z + 2r + l,i + fc — 2], and |ret„(X)| > d. Dinh's theorem says that ret„(A'„) = 
jr(^pd,k,n^^ the Set of facets of p^''^'". It is easy to check that when n = k, 
|ret„(X)| = d for all X € Xrn and that vetn{Xn) is the set of d-element Gale 
subsets of [0, fc], that is, the facets of the cyclic polytope P'^^^^^ , 

Note that Xn-i C Xn. We wish to describe jF(P'^''^'") in terms of 
jr(^pd,k,n-iy^ for this we need the following shift operations. If P = ret„_i(X) € 
jpjpd,fc,n~ij^ let the right-shift of P be rsh(P) = ret„(X + 1). Typically, 
rsh(P) contains P + 1. If it does not, then £ F \ X and rsh(P) = 
((P + 1) \ {!}) U {0}. In either case, |rsh(P)| > |P[ > d. If P = ret„(X) G 
jrl^pd^n^^^ let the left-shift of P be Ish(P) = ret„_i(X - 1). Note that 
Ish(P) \ {0} = (P - 1) n [1, n]; Ish(P) contains if either or 1 is in P. 

Lemma 3.2 Ifn>k+l and F £ r(pd^k,n) ^^^f^ maxP > k, then Ish(F) G 

d,k,n-l\ 
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Proof: Let F = ret„(X), with X = [i,i + 2r - l\UY U[i + k,i + k + 2r - 1]. 
Then X — 1 also has the form of equation (for i — 1). The set Ish(F) is 
the vertex set of a facet of p'^'^''^-^ as long as |lsh(i^)| > d. We check this 
in three cases. 

Case 1. Ifk <i + k + 2r-l <n, then i + 2r - 1 > 0, so Y C [i + 2r + 1, i + 
k-2]C[2,i + k-2]. Then 

Ish(F) D max{i + 2r - 2,0} U {Y - I) U [i + k - l,i + k + 2r - 2], 

so |lsh(F)| >l + {d-2r-l) + 2r = d. 

Case 2. If i+k > n, then i > n — k > 1. Also, |F| > d implies maxl" < n — 1. 
So 

Ish(F) = [i - 1, i + 2r - 2] U (y - 1) U {n - 1}, 
so |lsh(F)| = 2r + {d-2r-l) + l = d. 

Case 3. lfi + k<n<i + k + 2r-l, then i + 2r-l>n-k>l, and 
F = [max{0,i},i + 2r -l]UYU[i + k,n], 

so 

|F| = {i + 2r - max{0, i}) + {d - 2r - I) + {n - i - k + 1) 
= d + n — k — max{i, 0} > d+1. 

Then |lsh(F)| > |F[ - 1 > d 

Thus, Ish(F) is a facet of pdM,n-i_ □ 

Identify each facet of the ordinary polytope P'^^^'^ with its ordered list 
of vertices. Then order the facets of P'^''^'" in colex order. This means, if 
F = iii2 . . .ip and G = jij2 . . . jq, then F G if and only if for some t > 0, 
ip^t < jq-t while for < s < t, ips = jq-s- 

Lemma 3.3 Ifn > k+1 and Fi and F2 are facets of P'^'^'^ with max Fi > k, 
then Fi -<c F2 implies Ish(Fi) -<c lsh{F2). 

Proof: Suppose Fi -<c F2, and let q be the maximum vertex in F2 not in 
Fi. Then Ish(Fi) -<c lsh(i<2) as long as q > 2, for in that case q — 1 (z 
lsh(F2) \ Ish(Fi), while [q,n- 1] n Ish(Fi) = [q,n-l]n lsh(F2). (If q = 1, 
then q shifts to in lsh(i<2), but may be in Ish(Fi) as a shift of a smaller 
element.) So we prove q >2. Write 

F2 = retn([i, i + 2r - 1]UY U [i + k,i + k + 2r - 1]) 
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and 

Fi = retn([?', i' + 2r' - 1]UY' U[i' + k,i' + k + 2r' - 1]). 

Since maxF2 > h% i + 2r - 1 > 0, so Y U [i + k,i + k + 2r - 1] C 
[2,n], Thus, if g G Y U [i + k,i + k + 2r - 1], then q > 2. Otherwise 
Yu[i + k,i + k + 2r-l]) = Y'u[i' + k,i' + k + 2r' - 1]), but F / F'. This 
can only happen when Y U [i + k,i + k + 2r — 1]) is an interval; in this case 
i+k+2r-l > n+1. Then g = i+2r-l = {i + k+2r-l)-k > n+l-k > 2. 
□ 



Proposition 3.4 Let n> k + 1. The facets of P'^''^'^ are 

{F :F e ^(P'i.M-i) and maxF < n - 2} 

U {rsh{F) : F G J^(pd,k,n-i-j ^nd maxF > n - 2}. 

Proof: If maxX < n — 2, then ret„(X) = ret„_i(X); in this case, let- 
ting F = retn(X), F G j^(^pd,k,n-i^ if and only if F G T{P'^'^'''). If 
F G jr(P'i.*:."-i) with maxF > n - 2, then rsh(F) G J^{pdAn^ with 
maxrsh(F) > n — 1. Now suppose that G = ret„(X) G jc-(prf,fc,n) ^^^j^ 
maxG > n - 1. Let F = Ish(G) = ret„_i(X - 1) G jrj^pd,k,n-iy^ ^y^^^ 
maxF >n-2. By definition, rsh(F) = ret„((X - 1) + 1) = retn(X) = G. 
□ 



Theorem 3.5 Let Fi, F2, . . . , Fy he the facets of P^^^'"' in colex order. 
Then 

1. Fi, F2, . . . , Fy is a shelling of P'^'^'^. 

2. For each j there is a unique minimal face Gj of Fj not contained in 

3. For each j, 2 < j < v — 1, Gj contains the vertex of Fj of maximum 
index, and is contained in the d — 1 highest vertices of Fj . 

4- For each j, the face Gj is a simplex and the interval [Gj,Fj] is a 
Boolean lattice. 

Note that this theorem is not saying that the faces of P'^''^'" in the interval 
[Gj,Fj] are all simplices. 

Proof: We construct explicitly the faces Gj in terms of Fj . The reader may 
wish to refer to the example that follows the proof. 
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Cyclic polytopes. We start with the cyclic polytopes. (For the cyclics, 
the theorem is generally known, or at least a shorter proof based on |5] is 
possible, but we will need the description of the faces Gj later.) 

Let Fi, F2, Fy be the facets, in colex order, of P'^^^'^ ^ the cyclic 
d-polytope with vertex set [0,A;]. Each facet Fj can be written as Fj = 

U /| U /? U • • • U U 7.^, where is the interval of Fj containing 0, 

J J J J J J 

if G Fj, and = otherwise; is the interval of Fj containing k, if 
k & Fj, and Ij = otherwise; and the Ij are the other (even) intervals of 
Fj with the elements of Ij preceding the elements of Ij~^^- (For example, in 
P7'9'9, Fe = {0,1,2,4,5,7,8}, 1° = {0,1,2}, Ji = {4,5}, /| = {7,8}, and 
Iq = 0.) For the interval [a,b], write E{[a,b]) for the integers in the even 
positions in the interval, that is, E{[a, b]) = [a, b] (1 {a + 2i + 1 : i £ N}. Let 
Gj = U^^^^E{lf) U If. Since 1° = Fj if and only if j = 1, Gi = 0, and for 
all j > 1, Gj contains the maximum vertex of Fj. Since Fj is a simplex, 
[Gj,Fj] is a Boolean lattice. 

To show that Fi, F2, . . . , F^, is a shelling of P'^'''''^ we show that Gj is 
not in a facet before Fj and that every ridge of P'^^^'^ in Fj that does not 
contain Gj is contained in a previous facet. For j > the face Gj consists 
of the right end-set l'- (if nonempty) and the set Uj^]^F(/j) of singletons. 
Note that Gj is contained in the d — 1 highest vertices of Fj (equivalently, 
Gj does not contain the minimum element of Fj) unless j = v, in which case 
Gv = Fy. Any facet F of P'^'^^^ containing Gj must satisfy Gale's evenness 
condition and therefore must contain an integer adjacent to each element of 
U^^^F(/j). If any element of the form max/|-|-l is in F, then F occurs after 
Fj in colex order. This implies that any Fj previous to Fj and containing 
Gj also contains U^^-^Ij U Ij. But Fj is the first facet in colex order that 
contains U^^^Ij U Ij. So Gj is not in a facet before Fj. 

Now let g € Gj; we wish to show that Fj \ {g} is in a previous facet. If 
g G F(lJ) for £ > 0, let F = Fj \ {g} U {min/j - 1}. Then F satisfies Gale's 
evenness condition and is a facet before Fj. Otherwise 5 G /j \ E{lf); in 
this case let F = Fj \ {g} U {max/° + 1} (where we let max/^ + 1 = if 
Ij = 0). Again F satisfies Gale's evenness condition and is a facet before 

Thus the colex order of facets is a shelling order for the cyclic polytope 
pd,k,k^ and we have an explicit description for the minimal new face Gj as 
Fj is shelled on. 

General ordinary. Now we prove the theorem for general F'^-'^'" by 
induction on n > A;, for fixed k. Among the facets of P'^''^'"^ first in colex 
order are those with maximum vertex at most n — 2. These are also the 
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first facets in colex order of p'^-'^'"-!. Thus tlie induction hypothesis gives 
us that this initial segment is a partial shelling of P'^'^'"^ and that assertions 
2-4 hold for these facets. 

Later facets. It remains to consider the facets of P'^-^'" ending in n — 1 
or n. These facets come from shifting facets of P'^-'^'"-! ending in n — 2 or 
n — 1. Our strategy here will be to prove statement [21 of the theorem for 
these facets. The intersection of Fj with u|^|-Fi is then the antistar of Gj 
in Fj, and so it is the union of (d — 2)-faces that form an initial segment of 
a shelling of Fj . This will prove that the colex order Fi , F2 , . . . , F^, is a 
shelling of P'^-'^'". 

Note that there is nothing to show for the last facet of P"''^'" in colex 
order. It is F„ = [n — d + l,n], and is the only facet (other than the first) 
whose vertex set forms a single interval. Assume from now on that j is fixed, 
with j < V — 1. Later we will describe recursively the minimal new face Gj 
as Fj is shelled on. It will always be the case that max Fj G Gj. We will 
prove that Gj is truly a new face (is not contained in a previous facet), and 
that every ridge not containing all of Gj is contained in a previous facet. 

Ridges not containing the last vertex. It is convenient to start by 
showing that every ridge of P'^'^^^ contained in Fj and not containing max Fj 
is contained in an earlier facet. This case does not use the recursion needed 
for the other parts of the proof. Write 

X = + 2r - 1\\JY \J [i + k,i + k + 2r - I] 

and Fj = ret„(X) = {zi, Z2, ■ ■ ■ , Zp} with < zi < Z2 < • • • < Zp < n. The 
facet Fj is a (d — l)-multiplex, so its facets are of the form 

Fj{zt) = {ze : 1 < £ < p, < \£ - t\ < d - 2} 

for 2 < t < p-1, Fj{zi) = {zi,Z2, ■ ■ ■ ,Zd-i}, and Fj{zp) = {zp_d+2, ■ ■ ■ ,Zp^i,Zp}. 
If Fj{zt) does not contain max Fj = Zp, then t < p — d + 1 and this implies 
i<zt<i + 2r — 1. Consider such a Zt- 

The first ridge. For t = 1, there are three cases to consider. 
Case 1. Suppose zi > 1. Then Fj{zi) = + 2r — 1] U Y. Let / be the 
right-most interval of Fj{zi). Let Z = {I — k) L) Fj{zi), and F = ietn{Z). 
Since i > 1 and maxFj{zi) < i + k — 2, the interval I — k contributes at 
least one new element to F, so |-F| > d. 

Case 2. Suppose zi = and the right-most interval of Fj{zi) is odd. In 
this case the left-most interval of Fj must also be odd, so i < 0, and Fj{zi) 
contains i + k but not i + k — 1. Let F = Fj{zi) VJ {i + k — 1}. 
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Case 3. Suppose zi = and the right-most interval of Fj{zi) is even (and 
then so is the left-most interval). Then Fj{zi) = [0, i-|-2r— l]uyu[i-|-A;, k—l\ 
(where the last interval is empty if z = 0). Let 

F = Fj{zi) Li{i + 2r} = {0} U + 2r]UY U [i + k,k - 1]. 

(When i = and r = (d - l)/2, this gives F=[0,d- 1].) 

In all cases F is a facet of p^'*^'" containing Fj{zi). It does not contain 
max Fj, so F -<c Fj. 

Deleting a later vertex. Now assume 2<t<p — d + 1; then zt > 
max{i +1,1}. Here 

Fj{zt) = [max{i, 0}, zt-l]U [zt+i,i + 2r - l]UY U [i + k, zt - I + k], 

and |Fj(£t)| = zt — max{z, 0} + d — 2>d — 1. Also note that — 1 -|- A; is 
the {d — 2)nd element of {zi,Z2, ■ ■ ■ , Zp} after zt, so zt — 1 + k = zt+d-2 < 
Zp = max Fj . 

Case 1. li Zf — i is even, let F = Fj{zt) U {i + 2r}. Then F = ret„(Z), where 

Z = [i, Zt - l]U [zt + + 2r]UYU[i + k,zt- I + k], 
and \F\ > d. 

Case 2. lizt-i is odd and max([z,z-F2r-l]uy) < i+k-2, let F = ret„(.Z), 
where 

Z =[i-l,zt-l]l}[zt + l,i + 2r -l]l]Yl][i + k-l,zt-l + k]. 

Then F D Fj{zt) U {i + - 1}, so |F| > d. 

Case 3. Finally, suppose Zt — i\s odd and max Y = i + k — 2. Let [q,i + k — 2] 
be the right-most interval of Y, and let F = ret„(Z), where 

Z = [q - k,zt - l]V^[zt + \,i + 2r - {Y \ [q,i + k - 2]) U [g, - 1 + A;]. 

Then F D Fj{zt) U {i + - 1}, so |F| > d. 

In all cases, F is a facet of P'^'^^'^ containing Fj{zt) and maxFj ^ F, so 
F occurs before Fj in colex order. 

Determining the minimal new face. We now describe the faces Gj 
recursively. (We are still assuming that maxFj > n — 1.) Let G be the face 
of Ish(Fj) that is the minimal new face when Ish(Fj) is shelled on, in the 
colex shehing of the polytope p*^'*^-"-!. Let Gj = G + 1; then Gj contains 
maxPj. Also, Gj is a subset of the last d — 1 vertices of Pj, which form a 
facet of Fj] hence Gj is a face of Fj. For any facet Fi of P'^'^'^^ Gj C Fj if 
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and only if G C lsh(i<i). So by the induction hypothesis, Gj is not contained 
in a facet occurring before Fj in colex order. 

Ridges in previous facets. It remains to show that any ridge of P'^'^^^ 
contained in Fj but not containing all of Gj is contained in a facet prior to 
Fj . Note that we have already dealt with those ridges not containing max Fj . 
Now let g G G , gj = g + 1 G Gj , and assume gj ^ maxFj. The only ridge of 
pd,k,n contained in Fj, containing maxFj, and not containing gj is Fj{gj). 

Let H be the unique ridge of p^-'^''^-^ in Ish(Fj) containing max(lsh(i^j)), 
but not containing g. By the induction hypothesis, H is contained in a facet 
F of pd-'^'"^-^ occurring before lsh{Fj) in colex order. Suppose Fj{gj) is 
contained in a facet Fi of P'^-'^'" occurring after Fj in colex order. Then H 
is contained in lsh(i<£). Thus the ridge H of p'^'^^^-^ ig contained in three 
different facets: F (occurring before \s\i{Fj) in colex order), Ish(Fj), and 
\sh.{Fi) (occurring after Ish(Fj) in colex order). This contradiction shows 
that the ridge Fj{gj) can only be contained in a facet of P'^^^'"- occurring 
before Fj in colex order. 

Boolean intervals. Finally to verify assertion 4 of the theorem, observe 
that every facet Fj is a (d — l)-dimensional multiplex. The vertices of Gj 
are among the last d vertices of Fj and so are affinely independent 0; thus 
Gj is a simplex. The maximum vertex u of Fj is contained in Gj. The 
vertex figure of the maximum vertex in any multiplex is a simplex jH). So 
the interval [Gj,Fj], which is an interval in [u,Fj], is a Boolean lattice. □ 

A nonrecursive description of the faces Gj, generalizing that for the 
cyclic case in the proof, is as follows. Write the facet Fj as a disjoint union, 
Fj = A'j U Ij U Ij U ■ ■ ■ U Ij U Ij-, where is the interval of Fj containing n 
if n G Fj, and I'j = ^ otherwise; the Ij{l<£<p) are even intervals of Fj 
written in increasing order; and is 

• the interval containing 0, if maxFj < k — 1; 

• the union of the interval containing max Fj — k and the interval con- 
taining maxi^'j — k + 2 (if the latter exists), if k < max Fj < n — 1; 

• the interval containing n — k, if maxFj = n and n — k ^ Fj] 

• 0, if maxi^'j = n and n — k ^ Fj. 
Then Gj = Uf=i^(/j) U If. 

Example. Table Ogives the faces Fj and Gj for the colex shelling of 
the ordinary polytope p5,6,8 
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01 4567 
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012 5678 


5678 


8 


2345 8 


8 


16 


45678 


45678 



Table 1: Shelling of P^'^'' 



Let us look at what happens when facet Fi^ is shelled on. The ridges 
of p5,6,8 contained in F13 are 0123, 0236, 01367, 012678, 12378, 2368, and 
3678. The first ridge, 0123, is contained in Fi = 01234. The ridge 0236 is 
^^13(^2) = -^13(1), and max{[i,i + 2r-l]UY) = 3 < 4 = i + A: - 2, so we find 
that 0236 is contained in F4 = 02356. The ridge 01367 is ^13(23) = ^13(2), 
so we find that 01367 is contained in Fq = 013467. This facet F13 = 0123678 
is shifted from the facet 012567 of P^'^''', which in turn is shifted from the 
facet 01456 of the cychc polytope p5'6,6_ y^i^^^ oi456 occurs in the shelling 
of the cyclic polytope, its minimal new face is its right interval, 456. In 
p5,6,8^ then, the minimal new face when Fi^ is shelled on is 678. The other 
ridges of -F13 not containing 678 are 12378 and 2368. The interval [G13, F13] 
contains the triangle 678, the 3-simplex 3678, the 3-multiplex 012678, and 
Fi3 itself (which is a pyramid over 012678). 

Note that for the multiplex, Af^'" = p"^'^-"^ this theorem gives a shelling 
different from the one mentioned in Section [2 In the standard notation for 
the facets of the multiplex (see Definition 0]) , the colex shelling order is Fq, 

Fn-d+1, Fn- The statements of this section 
hold also for even-dimensional multiplexes. 

4 The /i-vector from the sheUing 

The /i- vector of a simplicial polytope can be obtained easily from any shelling 
of the polytope. For P a simplicial polytope, and U[Gj,Fj] the partition of 
a face lattice of P arising from a shelling, h{P, x) = J2j x'^~l'^^L For general 
polytopes, the (toric) /i-vector can also be decomposed according to the 
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shelling partition. For a shelling, Fi, F2, Fn, of a polytope P, write 
Qj for the set of faces of Fj not in Uj<jFj. Then h{P,x) = Z]^=i ^(^j) ^)) 
where h{Qj,x) = J^GeGj 9{G,x){x — i^^^-i-dimG^ However, in general we do 
not know that the coefficients of h{Qj,x) count anything natural, nor even 
that they are nonnegative. Stanley raised this issue in ^14i Section 6]. It has 
apparently been settled by Tom Braden [H]. 

We turn now to /i- vectors of ordinary polytopes. In pi we used the flag 
vector of the ordinary polytope to compute its toric /i- vector. 

Theorem 4.1 (j^j) For n>k>d = 2m + 1>5 and I <i <m, 

1 fT^dkr,\ (k-d + i\ , ^.(k-d + i-l\ 

We did not understand why the /i-vector turned out to have such a nice form. 
Here we show how the /i-vector can be computed from the colex shelling. 
Properties 2 and 4 of Theorem 13 . 51 are critical. 

In [HI we showed that the flag vector of a multiplicial polytope depends 
only on the /-vector. However, for our purposes here it is more useful to 
write the /i- vector in terms of the /-vector and the flag vector entries of the 
form /oi (the sum of the number of vertices on the z-faces) We introduce a 
modified /-vector. Let /_i = /_i = 1, /g = /q, and Jd-i = fd-i + (/o,d-i - 
dfd-i); and for 1 < j < d — 2, let 

fj = fj + ihj+i - U + 2)/j+i) + ihj - U + 



Theorem 4.2 If P is a multiplicial d-polytope, then 

h{p,x) = j2HPy-' = jzh-iiP){^ - 1)'-'- 

Proof: As observed in the proof of Theorem 12.11 the (7-polynomial of an 
e-dimensional multiplex M with n + 1 vertices is g{M, x) = l + {n — e)x. So 
for a multiplicial d-polytope P, 



h{p,x) = J2 g{G,x){x-iy 



-l-dimG 



G face of P 
Gj^P 

J2 (1 + (/o(G) - 1 - dimG)x)(x - l)^-i-dimG 

G face of P 
G^P 
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d d-1 



J2 h-i{x - if'' + Y.Uo^ - (^ + l)fi)x{x - lY'^- 



1=0 i=l 

d d-1 



^ - 1)^-^ + E(/o^ - + m)[{x - 1)"-' + {x- if-'-'] 

i=0 i=l 

{x-lf + Mx-lf-' 

d-1 

+ + {foi + l)f^) + (/o,i-l - ifi~l))ix - if-' 

i=2 

+ {fd-1 + ihd-i - dfd-i)) 
J2fi~i{P){x-lf'\ 

1=0 

□ 



Simplicial polytopes are a special case of multiplicial polytopes. Clearly, 
when P is simplicial, f{P) = f{P), and we recover the definition of the sim- 
plicial /i- vector in terms of the /-vector. The multiplicial /i- vector formula 
can be thought of as breaking into two parts: one involving the /-vector, 
and matching the simplicial /i-vector formula; the other involving the "ex- 
cess vertex counts," foj — {j + l)/j. In the simplicial case the sum of the 
entries in the /i-vector is the number of facets. For multiplicial polytopes 

Eto hiiP) = fd-i{P) = fd-i + (/o,d-i - dfd-i). 

In general, applying the simplicial /i-formula to a nonsimplicial /-vector 
produces a vector with no (known) combinatorial interpretation. This vector 
is neither symmetric nor nonnegative in general. We will see that in the 
case of ordinary polytopes something special happens. Write h'{P,x) = 
J2i=o h[{P)x'^~'^ for the /i-polynomial that P would have if it were simplicial, 
by letting h[{P) be the coefficient of x'^"* in Ya=o fi-i{P){x - if-'. 

Theorem 4.3 Let P'^-'^'" he an ordinary polytope. Let U^^i[Gj, Fj] be the 
partition of the face lattice of P'^'^'^ associated with the colex shelling of 
pd,k,n_ r^j^^^ alii, 0<i< d, h'{P'^'^''^, x) = Yfj=i x'^-\^^\. 

Furthermore, if C^'^ is the cyclic d-polytope with k+l vertices, then for 
alli,0<i< d, h'iiP'^'^^'') > hiiC^'^), with equality for i > d/2. 

Proof: Direct evaluation gives /io(P) = h'^{P) = I- Let Fi, F2, . . . , F^, be 
the colex shelling of P'^-'^'". By Theorem l3.5l part 2, the set of faces of P'^^^^"- 
has a partition as U'j^ilGj, Fj]. By Theorem l3.5l part 4, the interval [Gj, Fj] 
has exactly C^7-di'mG'^0 faces of dimension i for dimGj < £ < d — I. Let 
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ki = \{j ■ dimGj = i — 1}\ = \{j : \Gj\ = i}\ (since all the Gj are simplices). 
Then fi = X]i=o ie-J+ij^i- These are the (invertible) equations that give fi 
in terms of h'^, so for all i, h[ = ki = \{j : \Gj\ = i}\. 

The second part we prove by induction on n > k. Recall from the 
proof of Theorem 13.51 that for each facet Fj of P'^'^^'^^ Gj is the same size 
as the minimum new face G of the corresponding facet of p<^''^'"~i; that 
facet is the same (as vertex set) as Fj, if maxFj < n — 2, and is Ish(Fj), 
if maxFj > n — 1. From Proposition 13.41 we see that each facet of P'^-'^'"-! 
with maximum vertex n — 2 gives rise to two facets of P'^^^''^^ while all others 
give rise to exactly one facet each. So for all i, 

h[{pdAn^ = /i^(p'^.^."-i) 

+ \{j : Fj is a facet of P'^^^''^ with maxFj = n — 1 and = i}]. 

Thus, h[{P'^^^''') > /i^(P'^'*='"-i), so by induction, /i^(P'^''^'") > /i^(C"^''^') for 
all i. 

Now consider Fj with max Fj = n — 1. In the description of the faces 
Gj following the proof of Theorem E31 If = 9 and Fj \ A'^j Q Y U [i + k,i + 
k + 2r-l]. Thus \Gj\ < \E{Y U [i + k,i + k + 2r - = (d-l)/2. So for 
i > d/2, /i^(P'^'*^'") = h'iiP'^^^^'^-^) = hiiC^^^). a 

Note that for the multiplex M'^'" {d odd or even), /i'(M'^'") = {l,n-d + 
1,1,1,... ,1,1), while /i(M'^'") = (l,n-(i+l,n-d+l,...,n-(i+l,l). 

Now for multiplicial polytopes, we consider the remaining part of the 
/i- vector, coming from the parameters foj — (j + l)/j. This is 

h{P,x) - h'{P,x) 

d-1 

= ifo,d-i - dh-i) + E ((/o,^ - + + ih^-l - ifi-i)) - 1)'-^ 

1=2 

So 

h{P,x + l) - h'{P,x + l) 

d-1 

= ihd^l - dfd-l) + E (ih^ - + + ih^-l - ifi-l)) X"-' 

i=2 

= E(/o,i-(i + l)/i)(x + l)x'^-^-\ 

i=2 

^° d 1 d 1 

J2{h{p) - h[{p)){x + 1)^-1- = Y.Uo, + i)h)x'-'-\ 

1=2 i=2 
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For the ordinary polytope, this equation can be apphed locally to give 
the contribution to h{P'^'^'^ ,x) — h' [P'^''^'"- , x) from each interval [Gj,Fj] 
of the shelling partition. For each j, and each i > dimGj, let bj^i = 
J2(.foiH) — {i + 1)), where the sum is over all i-faces H in [Gj,Fj]. Let 
:dimG bj,i^'^ ^ *• Write bj{x) in the basis of powers of (x + 1): 
bj{x) = J2 o-jA^ + Then Oj^j = hi{Gj) — h[{Qj), the contribution to 

hi{P'^'^'"') - h'iiP'^'''^'') from faces in the interval [Gj,Fj]. Note that for fixed 
J2iO'j,i = ^j(O) = fo{Fj) — d. We will return to the coefficients aj^i after 
triangulating the ordinary polytope. 

Example. The /i-vector of P^'^'^ is h{P^'^'^) = (1,4,7,7,4,1). The 
sum of the hi is 24, which counts the 16 facets plus one for each of the 
four 6-vertex facets, plus two for each of the two 7-vertex facets. Referring 
to Table 1, we see that h'{P^'^'^) = (1,4,5,3,2,1); from this we compute 
y^p5,6,8^ _ 3]^^ 44^ 16). The nonzero aj^i here are a6,2 = ^7,3 = ^11,2 = 
fli2,3 = 1 and ai3^3 = 015^4 = 2. In this case each interval [Gj, Fj] contributes 
to hi{P'^'^''^) — h[{P'^'^''^) for at most one i, but this is not true in general. 

5 Triangulating the ordinary polytope 

Triangulations of polytopes or of their boundaries can be used to calculate 
the /i- vector of the polytope if the triangulation is shallow |2j. The solid 
ordinary polytope need not have a shallow triangulation, but its boundary 
does have a shallow triangulation. The triangulation is obtained simply by 
triangulating each multiplex as in Section |21 This triangulation is obtained 
by pushing the vertices in the order 0, 1, . . . , n. 

Theorem 5.1 The boundary of the ordinary polytope P'^'^'"- has a shallow 
triangulation. The facets of one such triangulation are the Gale subsets of 
[i, i + k] (where < i < n — k) of size d containing either or n or the set 
{i,i + k}. 

Proof: First we show that each such set is a consecutive subset of some 
facet of pd-'^'"^. Suppose Z is a Gale subset of [i,i + k] of size d containing 
{i,i + k}. Write Z = [i,i + a - 1]UY U [i + k - b + l,i + k], where a > 1, 

6 > 1, and y fl {i + a, i + /c — 6} = 0. Since Z is a Gale subset, \Y\ is even; 
let r = (d — 1 — |y|)/2. Since \Z\ = d, a + 6 = 2r + 1, so a and b are each at 
most 2r. Define X = [i + a - 2r,i + a - l]UY U[i + k - b+ l,i + k - b + 2r]. 
Note that i + k — b + 1 = (i + a — 2r) + k. Then ret„(X) is the vertex set of 
a facet of P'^'^^^^ and Z is a consecutive subset of retn{X). 
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Now suppose that Z is a Gale subset of [0, k] of size d containing 0, 
but not k. Write Z = {0} U Y U [j - 2r + where j < k, r > 1, 

and j — 2r Y. Then \Y\ = d — 2r — 1, and Z = ret„(X), where X = 
[j — 2r + I — k, j — k] L) Y L) [j — 2r + 1, j]. So Z itself is the vertex set of a 
facet of P'^'^^^, The case of sets containing n but not n — k works the same 
way. 

Next we show that all consecutive d-subsets of facets F of P'^'^'" are of 
one of these types. Let F = ret„(X), where X = [i^i + 2r — 1\UY \j[i + k,i + 
/c + 2r — 1] , with Y a paired subset of size d — 2r — 1 of [i + 2r + 1, i + k — 2]. 
Suppose first that i + 2r — 1 > and i + k < n. Let Z he a consecutive 
d-subset of F. Since \Y\ = d - 2r - 1, \[i,i + 2r - 1] n F\ < 2r, and 
\[i + k,i + k + 2r — l]r)F\ < 2r, it follows that i + 2r — 1 and i + k must both 
be in Z. Thus we can write Z = [i + 2r — a,i + 2r — l]L)YL)[i + k,i + k + b—l], 
with a + 6 = 2r + 1, i + 2r — a > 0, and i + k + b — 1 < n. Let i = i + 2r — a. 
Then i + k + b — l = i + k, so 0<i<n — k, and Z is a Gale subset of 
[i, £ + k] containing {£, i + k}. 

If i + 2r - 1 < 0, then i + k + 2r-l<k<n, and F = {0}UYU[i + 
k,i + k -\- 2r — 1]. Then \F\ = d and F itself is a Gale subset of [0, k] of size 
d containing 0. Similarly for the case i + k > n. 

The sets described are exactly the {d — l)-simplices obtained by trian- 
gulating each facet of P"^'*^'"- according to Theorem 12.11 The fact that this 
triangulation is shallow follows from the corresponding fact for the triangu- 
lations of the multiplexes. □ 

Let T = be this triangulation of dP'^''''^. Since T is shallow, 

h(pdAn^^^ = h{T,x). We calculate h{T,x) by shelling T. 

Theorem 5.2 Let Fi, F2, . . . , Fy be the colex order of the facets of P'^''^''^. 
For each j, if Fj = {zi,Z2, ■ ■ ■ , Zp^} (zi < Z2 < ■ ■ ■ < Zp^), and 1 < i < 
Pj - d + I, let Tj^i = {zi, Zi+i, ... , ze+d-i}- Then Ti^i, Ti^2, ■ ■ ■ , 7i,pi-d+i, 
72,1; • • • , T2,p2-d+i, Ty,i, Ty,p,-d+i is a shelling of T{P'^'^'"-). 

Let Uj/ be the minimal new face when Tj^i is shelled on. As vertex sets, 
Uj,pj-d+i = Gj. 

Proof: Throughout the proof, write Fj = {zi, Z2, ■ ■ ■ , Zp.} {zi < Z2 < ■ • • < 
Zpj). We first show that Gj is the unique minimal face of Tj^p.-d+i not 

contained in {uj^l U^^i'^^^ Ti^^) U (U^ii'^rj,^). The set Gj is not contained 
in a facet of P'^^^'"- earlier than Fj. So Gj does not occur in a facet of T of 
the form Tj,£ for i < j. Also, maxFj S Gj, so Gj does not occur in a facet 
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of T of the form Tj^g^ for i < pj — d. Thus Gj does not occur in a facet of T 

before Tj.p^_d+i- 

We show that for Zq S Gj, Tj^p^-d+i \ {zq} is contained in a facet of T 
occurring before Tj^p^_(i^i. There is nothing to check for j = v, because 
Tv,p„-d+i = is the last simplex in the purported shelling order. So we 
may assume that j < v and thus Gj is contained in the last d — 1 vertices 
ofF,. 

Case 1. 11 Pj > d and g = pj (zg is the maximal element of Fj), then 
Tj,p^-d+i \ {zpj} C Tj^p.-d,. 

Case 2. Suppose pj — d + 2 < q < pj — 1. Then Tj^p._d+i \ {zq} C 
{zq-d+2i ■■ ■ , Zq-i,Zq^i . . . , Zp^} = H. This is a ridge of P'^'^'"- in Fj not 
containing Gj, and hence H is contained in a previous facet F£ of P'^''^'". 
Since H is a ridge in both Fj and Fg, H is obtained from each facet by 
deleting a single clement from a consecutive string of vertices in the facet. 
So \H\ < \Fir\[zq_d+2, Zp^] \ < l-H'l + l, and sod-1 < \Fin[zp^^d+i, Zpj]\ < d. 
So Tj^p.-fi+i \ {zq} is contained in a consecutive set of d elements of Fg, and 
hence in a (c?— l)-simplex of belonging to Fg. This simplex occurs 

before Tj^p^^a+i in the specified shelling order. 

Case 3. Otherwise pj = d (so pj — d + 1 = 1) and q = d. Then Tj^i = Fj 
and H = Tj^i \ {zd} is a ridge of P'^^^^^ in Fj not containing maxFj, so 
H is contained in a previous facet F^ of P<^'*^'". As in Case 2, d — \ < 
\Fg n [zijZd-i]! < d. So Tj^i \ {zd} is contained in a consecutive set of d 
elements of Fg, and hence in a (d— l)-simplex of 7'(P'^'^'"') belonging to Fg. 
This simplex occurs before Tj^p.^d+i in the specified shelling order. 

So in the potential shelling of T, Gj is the unique minimal new face 
as Tj^p.-d+\ is shelled on. Write Uj^p.-d+i = Gj. At this point we need 
a clearer view of the simplex T^ ^. Recall that Fj is of the form ret„(X), 
where X = [i,i + 2r - I] U Y U [i + k,i + k + 2r - I], with Y a subset of 
size d — 2r — 1. If i + 2r — 1 < or i + > n, then pj = \Fj\ = d, and 
Tj^i = Tj^p._d+i = Fj-, we have already completed this case. So assume 
i + 2r — 1 > and i + k < n. A consecutive string of length d in ret„(X) 
must then be of the form [i + s,i + 2r — 1] U y U [i + k,i + k + s] for 
some s, < s < 2r — 1. (All such strings — with appropriate Y — having 
i + s >0 and i + k + s < n occur as Tj^£.) In particular, for £ < pj — d + 1, 
Tj/ = Tjjjf-i \ {maxTj^^+i} U {minT^^^-i-i — 1} and maxTj,^ = minPj^^ + k. 

Now define Uj^£ for i < pj — d recursively by Uj^i = \ {z} U {z — 

k,z — 1}, where z = maxTj^^+i. By the observations above, Uj^g C Tj^g. 
We prove by downward induction that Uj^g is not contained in a facet Pj of 
pd,k,n ijgfQj-g p^.^ ^jiat Uj^g is not contained in a facet of T occurring before 
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Tj^£, and that any ridge of T in Tj^£ not containing all of Uj/ is in an earlier 
facet of T. The base case of the induction is i = pj — d + 1, and this case 
has been handled above. 

Note that {z — k,z — l} is a diagonal of the 2-face {z — k — l,z — k,z — l,z} 
of p*^'*^'"- [7]. So if Fi is a facet of P'^^^'^ containing then Fj contains 
{z — k — 1, z — z — 1, z}. Thus Fj contains so, by the induction 

assumption, i > j. Therefore, for i < j, and any r, Ti^r does not contain 
Uj^£. For r < £, Tj^r does not contain z — 1 = maxTj^i, so Tj^r does not 
contain Uj/. 

Now we wish to show that for any g G Uj^i, Tj^i \ {g} is in a previous 
facet of T. 

Case 1. li g = z — 1 = maxTj^^ and £ > 2, then T^-^^ \ {g} C Tj^£_i. 
Case 2. li g = z — 1 = max Tj_£ and i = 1, then Tj^^\{g} is the leftmost ridge 
of P'^'*^'" in Pj and, in particular, does not contain maxPj. So H = Tj £\{g} 
is contained in a previous facet Fe of As in the i = pj - d + I case, 

Fg n [minTj^^, maxTj^^] is contained in a consecutive set of d elements of Fe, 
and hence in a (d — l)-simplex of T(P'^'^'") belonging to Fg- So Tj £ \ {gr} is 
contained in a previous facet of T. 

Case 5. Suppose g < z — 1 and 5 G Uj^i fl J^j'^+i- Since {z — 1, z} C Tj^£^i, 
Tj^£^i contains at most d — 3 elements less than g. The ridge H of P'^-'^'" 
in Pj containing Tj^£^i \ {g} consists of the d — 2 elements of Fj below g 
and the (up to) d — 2 elements of Fj above g. In particular, H contains 
minT,_^+i - 1 = minTj^^. So Tj^£ \ {g} C H. Since dimT^^^ \ {g} = d - 2, 
H is the (unique) smallest face of P'^^^'"- containing Tj.^+i \ {g}- By the 
induction hypothesis F,- £+1 \ {g} is contained in a previous facet Ti^r of T; 
here i < j because maxTj^^+i e Tj^£+i \ {g}. The (d — 2)-simplex Tj^^+i \ {5} 
is then contained in a ridge of P'^-'^'" contained in Pj, but this ridge must be 
H, by the uniqueness of H. So Tj ^ \ {(7} C H = FiCiFj. As in earlier cases, 
Fi n [minTj^^, maxTj^^] is contained in a consecutive set of d elements of Pj, 
and hence in a (d — l)-simplex of T{P'^'^'^) belonging to Pj. So Tj ^ \ {g} is 
contained in a previous facet of T. 

Case 4- Finally, let g = z — k, which is minTj_^ + 1. Then Tj^£ contains d — 2 
elements above g. Let H be the ridge of P'^-'^'" in Fj containing Tj^£ \ {g}. 
Then maxP = max Tj£ < maxPj, so H does not contain Gj. So H is in 
a previous facet Pj of p'^^^'^. As in earlier cases, Pj fl [min Tj^^, max Tj^^] is 
contained in a consecutive set of d elements of Pj, and hence in a (d — 1)- 
simplex of P(P'^''^'") belonging to P^. So Tj^£ \ {g} is contained in a previous 
facet of T. 

Thus Ti^i, ri_2, • • • , Ti pj„rf+i, T2,i, . . . , T2^p2_rf+i, . . . , T^,^i, . . . 
is a shelling of T(P'^''''"). □ 
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Corollary 5.3 Let n > k > d = 2m + 1 > 5. Let U[Gj, Fj] be the partition 
of the face lattice of P'^'^'"' from the colex shelling, and let \j\Uj^i,Tj^(\ be the 
partition of the face lattice of T{P^'^''^) from the shelling of Theorem \5.iA 
Then 

2. The contribution to hi{P'^'^'^) — h[{P'^'^''^) from the interval [Gj,Fj] is 
aj,i = \{£ : \Uj^i\ =i,l<£<pe-d}\>0. 

Proof: The /i-vector of T = T(P'^''^'") counts the sets Uj^i of each size. 
Among these are all the sets Gj counted by the /i'-vector of P'^'*^'"-. Thus 

^ h,{T) = \{{j,l):\U,,e\ = i}\ 

> \{{j,e) : \Uj,e\ = i and £ = pj-d+ 1}\ = /^^(P'^'^'"). 

Recall that we write Qj for the set of faces of Fj not in Uj<j-Fi; here Qj is 
the set of faces in [Gj, Fj]. Write also TQj for the set of faces of T that are 
contained in Fj but not in Ui<^jFi. By [2 Corollary 7], since T is a shallow 
triangulation of QP'^'^'"', g{G,x) = j^^rf-i-dimo-^ where the sum is over 

all faces a of T that are contained in G but not in any proper subface of G. 
Thus 

pl-d+1 

Since h'{gj,x) = x'^-\^^\ = x'^~^^^-pj-''+'\ 

Pl-d 



Yoj^.x' = h{Gj,x) - h'{Gj,x) = Y x''"'^^'*', 

i 1=1 

aj,i = \{£ : \Uj^e\ = i, 1 < £ < pe - d}\ > 0. 



□ 



Example. Table|Hgives the shelling of the triangulation of p5,6,8 (^j^gfgj- 
back to Table^for the shelling of p^'^^^ itself.) Among the rows (6, 1), (7, 1), 
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1 9 1 
iZ, i 




AO I 


4,1 


23 56 


6 


12,2 


2 45 78 


578 


5,1 


3456 


46 


13,1 


0123 6 


126 


6,1 


01 34 6 


16 


13,2 


123 67 


267 


6,2 


1 34 67 


7 


13,3 


23 678 


678 


7,1 


01 456 


156 


14,1 


34 678 


4678 


7,2 


1 4567 


57 


15,1 


012 56 


1256 


8,1 


2345 8 


8 


15,2 


12 567 


2567 


9,1 


23 56 8 


68 


15,3 


2 5678 


5678 


10,1 


3456 8 


468 


16,1 


45678 


45678 



Table 2: Shelling of triangulatioii of p5,6,8 

(11,1), (12,1), (13,1), (13,2), (15,1), (15,2) (rows that are not the last 
row for that j), count the Uj^£ of cardinality i to get hi{P^'^'^) — h[{P^'^'^). 
Note that C/13,3 = G13 (from Tabled), and that C/13,2 = ?7i3,3 \ {8} U {2, 7}. 
The ridges in'Ti3,2 are 1236, 1237, 1267, 1367, and 2367. The first ridge, 
1236, falls under Case 1 of the (later part of the) proof of Theorem 15.21 it 
is contained in the previous facet, Ti3^i. The next ridge, 1237, falls under 
Case 3; it is contained in the ridge 12378 of P^'^'^ in F13 = 0123678, and 
12378 also contains the ridge 2378 in ri3^3. The induction assumption says 
that 2378 is contained in an earlier facet, in this case Tii^2, and 12378 is 
contained in Fn. Finally, the ridge 1237 is contained in the simplex Tn^i, 
part of the triangulation of Fn. The last ridge of Ti3^2 not containing 267 
is 1367. It falls under Case 4. The set 1367 is contained in the ridge 01367 
of p5,6,8^ contained in F13. This ridge is also contained in the earlier facet 
Fq. The ridge 1367 of the triangulation is contained in the simplex Tq^2- 

Theorem 5.4 Letn> d+k-l. Forl<i< d-1, /ii(P'^''''") -/ii(P'^''='"-i) 
is the number of facets Tj^£ of T{P'^''^'^) such that maxFj = n — 1 and 
\Uj^t\ = i. Forl<i<{d- l)/2, this is C'i^f^). 

Proof: Refer to Proposition 13.41 for a description of the facets of P'^^^'"- in 
terms of those of pd-^^'^-^_ For n > d + k — 1, for every facet P'^''^'" with 
maximum element n, the translation F — 1 is a facet of (For smaller 

n, a facet of P'^^^^^ may contain both and n, in which case Ish(F) is a proper 
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subset of F — 1.) The same holds for the simphces Tj^£ triangulating these 
facets, and for the sets Uj^g. The facets of P'^'^^^ with maximum element at 
most n — 2 are facets of p'^'^''^-^^ and the same holds for the corresponding 
Tj^£ and Uj^£. The contributions to h{P'^'^'^) from facets ending in any 
element but n - 1 thus total /i(P'^''='"-i). So for 1 < i < d - 1, hi{P'^'^^'') - 
f^^^pd,k,n-i^ is the number of facets Tj^i of 7'(P'^''^'") such that maxFj = n—l 
and \Uj^i\ = i. 

Now consider the set 5 of facets Tj^£ of 7"(P'^'^'"') with maxFj = n — 1. 
For each T G 5, T is a set of d elements occurring consecutively in some Fj 
with maximum element n — l. So T can be written as 

T =[b,n-k-l]U[n-k + l,c]UYU[e,b + k], (2) 

where 

1. n-k-d+l<b<n-k-l; 

2. n — k<c<b + d — 1 and c — n + k is even (here c = n — k means 
[n-k + l,c] =0); 

3. y is a paired subset of [c + 2, e — 1]; 

4. e = b + k — liin — k — b is odd, and e = 6+ A;ifn — A; — 6is even; and 

5. |r| = d. 

In these terms, the minimum new face U when T is shelled on is [/ = 
[6 + 1, n - A: - 1] U E{Y) U {b + k}. 

We give a bijection between the facets T in iS with \U\ = i (where 
I < i < {d — l)/2) and the {k — d)-element subsets of [1, A; — d + i — 1]. 
Let T be as in Equation |2l Then i = \U\ = n — k — b + 1^1/2. Write 
[c + 1, e — 1] \ y = {xi, X2, . • • , Xk^d}, with the x^s increasing. (This set has 
k — d elements because d = {c — b) + \Y\ + {b+k — e + l), so |[c+l, e — 1] \ y | = 
e — c — 1 — |y| = k — d.) For each i, let y{xi) be the number of pairs in Y 
with both elements less than X£. Let ai = n — k — b = i — |y|/2. Set 

A{T) = {ai + y{xi) + i - 1 : 1 < £ < k - d}. 

To see that this is a subset of [1, A; — d + z — 1], note that the elements of 
A{T) form an increasing sequence with minimum element oi and maximum 
element ai +y{xk-d) + {k - d - 1) < oi + |y|/2 + (A: - d- 1) = k-d + i-1. 

For the inverse of this map, write a (A;— d) -element subset of [1, k—d+i—1] 
as A = {ai, 02, ■ • • , flfe-d}, with the a^s increasing. Then 1 < oi < i. Let 

xi=n — k + d — 2i + ai— xi^-i odd). 



23 



Set 



T{A) = [n-k-ai,n-k-l]U[n-k + l,xi-l] 
UY U [n — ai — x(ai odd), n — ai], 

where 

Y =[xi,n-ai-l- x(ai odd)] \ {xi + 2(a£ - ai) - {£ - 1) : 1 < i < k - d}. 

We check that this gives a set of the required form. 

(1) Since 1 <ai <i<d — 1, n — k — d+l<n — k — ai < n — k — 1. 

(2) xi — 1 — n + k = d — 2i — 1 + {ai — x{cti odd)), which is nonnegative and 
even; xi — 1 = {n — k — ai + d—1) — {2i~2ai+x{o,i odd)) < n — k — ai + d—1. 

(3) Y is clearly a subset of [xi + l,n — ai — xi^^i odd) — 1]. To see that 

Y is paired, note that the difference between two consecutive elements in 
the removed set is {xi + 2(a^_|_i — ai) — £) — {xi + 2(a^ — ai) — {£ — 1)) = 
2{a£+i -ai) -1. 

(4) This condition holds by definition. 

(5) Since A is a subset of [1, A; — d + i — 1], 

xi + 2{ak-d - ai) - {k - d-1) 

< xi + 2{k - d + i - 1) -2ai - {k - d- I) 

= xi + k — d + 2i — 2ai — 1 = ra — ai — xi^^i odd) — 1. 

So 

{xi + 2(a^ - ai) - {£- 1) : 1 <£ < k - d} C [xi,n - ai - 1 - x{ai odd)], 
and 

|y| = {n — ai — x{(^i odd) — xi) — {k — d) = 2i — 2ai. 

So \T{A)\ =xi-{n-k-ai) + \Y\ + x{ai odd) = d. 

Also, in this case U = [n — k — ai + l,n — k — 1] Li E{Y) U {n — oi}, so 
\U\ = z. 

It is straightforward to check that these maps are inverses. The main 
point is that, if = ai + y{x£) + £ — 1, then 

xi + 2{ae-ai)-{e-l) = xi + 2{y{xe)+£-l)-{i-l) 

= xi + 2y{xi) + £-1 = Xi. 

□ 
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Example. Consider the ordinary polytope p"^'^'^^. There are six facets 
with maximum vertex 14; they are (with sets Gj underhned) {4, 5, 7, 8, 9, 10, 13, 14}, 
{4, 5, 7, 8, 10, n, 13, 14}, {4, 5, 8, 9, 10, 11, 13, 14}, {2, 3, 4, 5, 7, 8, 11, 12, 13, 14}, 
{2,3,4,5,8,9,11,12,13,14}, and {0, 1, 2, 3, 4, 5, 9, 10, 11, 12, 13, 14}- Among 
the 6-simphces occurring in the triangulation of these facets, six have \Uj/\ = 
3. Table|31gives the bijection from this set of simphces to the 2-element sub- 
sets of [1,4]. 
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Table 3: Bijection with 2-element subsets of {1,2,3,4} 

Again, the results of this section hold for even-dimensional multiplexes 
as well. 



6 Afterword 

The story of the combinatorics of simplicial polytopes is a beautiful one. 
There one finds an intricate interplay among the face lattice of the poly- 
tope, shellings, the Stanley-Reisner ring and the toric variety, tied together 
with the /i- vector. (See, for example, ^1 Lecture 8].) The cyclic poly- 
topes play a special role, serving as the extreme examples, and providing 
the environment in which to build representative polytopes for each /i- vector 
(the Billera-Lee construction [S]). In the general case of arbitrary convex 
polytopes, the various puzzle pieces have not interlocked as well. In this 
paper we made progress on putting the puzzle together for the special class 
of ordinary polytopes. Since the ordinary polytopes generalize the cyclic 
polytopes, a natural next step would be to mimic the Billera-Lee construc- 
tion, or Kalai's extension of it ^U], on the ordinary polytopes, as a way 
of generating multiplicial flag vectors. It would also be interesting to see 
if there is a ring associated with these polytopes, particularly one having 
a quotient with Hilbert function equal to the /I'-polynomial. Another open 
problem is to determine the best even-dimensional analogues of the ordinary 
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polytopes. They may come from taking vertex figures of odd-dimensional 

ordinary polytopes, or from generalizing Dinh's combinatorial description of 
the facets of ordinary polytopes. Looking beyond ordinary and multiplicial 
polytopes, we should ask what other classes of polytopes have shellings with 
special properties that relate to the ^-vector? 
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